Fingerprint image reading apparatus

ABSTRACT

An image reading apparatus which reads a fingerprint image, includes a roller rotatably mounted to the image reading apparatus, a given pattern being printed on part of an outer surface of the roller in a circumferential direction, a line sensor including a plurality of image pickup elements which read a fingerprint image from a finger that touches with the roller as well as the pattern printed on the roller, a generating unit which generates a reference value to each of the image pickup elements corresponding to part of the line sensor, which reads the pattern, based on image data of the pattern, a rotation sensing image extracting unit which extracts a rotation sensing image based on the reference value, and a determination unit which determines capture timing of the fingerprint image based on variations in the rotation sensing image.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based upon and claims the benefit of priorityfrom prior Japanese Patent Applications No. 2003-114831, filed Apr. 18,2003; and No. 2003-159657, filed Jun. 4, 2003, the entire contents ofboth of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] The present invention relates to an image reading apparatus forreading fingerprint image data.

[0004] 2. Description of the Related Art

[0005] Recently, a fingerprint recognition apparatus (fingerprint imagereading apparatus) has been used. This apparatus reads a fingerprint ofa person and recognizes it to identify the person.

[0006] A fingerprint recognition apparatus using a two-dimensional planesensor or a one-dimensional line sensor (one-dimensional image pickupdevice) is proposed. The apparatus using a one-dimensional line sensorhas a transparent, flat plate in a fingerprint reading position. A lightsource for illumination, a rod lens group (a SELFOC lens) and a linesensor are arranged under the transparent, flat plate.

[0007] Another fingerprint recognition apparatus using a line sensor isproposed in which the line sensor is incorporated into a hollow,transparent roller. In this apparatus, a user moves his or her fingerand presses it on the transparent roller. The line sensor reads afingerprint image of the pressed finger. A given printing pattern isprinted on the outer surface of an end portion of the transparentroller. The line sensor reads the printing pattern as well as thefingerprint image and senses variations in the printing pattern todetermine image data reading timing.

[0008] In general, an apparatus using a line sensor has to readreference white and black images and prepare adjustment data in advancein order to correct the influence of differences in characteristicsbetween image pickup elements and lens optical systems which configurethe line sensor.

[0009] To determine a read image by a predetermined reference value islikely to make the operation more unstable because the apparatus usingan optical image pickup element is affected by variations in outsidelight or variations in light source due to a decrease in batteryvoltage, even though it prepares correction data.

BRIEF SUMMARY OF THE INVENTION

[0010] According to an embodiment of the present invention, there isprovided an image reading apparatus which reads a fingerprint image,comprising:

[0011] a roller rotatably mounted to the image reading apparatus, agiven pattern being printed on part of an outer surface of the roller ina circumferential direction;

[0012] a line sensor including a plurality of image pickup elementswhich read a fingerprint image from a finger that touches with theroller as well as the pattern printed on the roller;

[0013] a generating unit which generates a reference value to each ofthe image pickup elements corresponding to part of the line sensor,which reads the pattern, based on image data of the pattern read by theline sensor;

[0014] a rotation sensing image extracting unit which extracts arotation sensing image based on the reference value generated by thegenerating unit; and

[0015] a determination unit which determines capture timing of thefingerprint image read by the line sensor based on variations in therotation sensing image extracted by the rotation sensing imageextracting unit.

[0016] According to another embodiment of the present invention, thereis provided an image processing apparatus comprising:

[0017] a line sensor including a plurality of image pickup elements;

[0018] a pixel value detecting unit which detects a first pixel valueand a second pixel value from each image data including a plurality ofpixels output from each the image pickup elements;

[0019] a pixel value range detecting unit which detects a pixel valuerange between the first pixel value and the second pixel value detectedby the pixel value detecting unit;

[0020] a normalized data generating unit which generates normalized datathat indicates a ratio of a pixel value of each of the pixels of theimage data to the pixel value range;

[0021] a normalized data average calculating unit which calculates anaverage of the normalized data generated by the normalized datagenerating unit; and

[0022] a pixel value correcting unit which corrects a pixel value ofeach of the pixels of the image data based on the average calculated bythe normalized data average calculating unit and the pixel value of eachof the pixels.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

[0023]FIG. 1 is a block diagram of an electronic circuit of a mobilephone with a fingerprint reading apparatus according to an embodiment ofthe present invention.

[0024]FIG. 2 is an external view of the mobile phone.

[0025]FIG. 3 is a schematic sectional view of a mechanical section ofthe fingerprint reading apparatus according to the embodiment of thepresent invention.

[0026]FIG. 4 is a perspective view showing an outward appearance of aroller with a printing pattern in the fingerprint reading apparatusaccording to the embodiment of the present invention.

[0027]FIG. 5 is a diagram showing a correspondence between the printingpattern of the roller and image pickup elements of a line sensor in thefingerprint reading apparatus according to the embodiment of the presentinvention.

[0028]FIG. 6 is a main flowchart of an image capture process performedby the fingerprint reading apparatus according to the embodiment of thepresent invention.

[0029]FIG. 7 is a flowchart of a correction data generation process inthe image capture process performed by the fingerprint reading apparatusaccording to the embodiment of the present invention.

[0030]FIG. 8 is a flowchart of a rotation sensing image extractionprocess in the image capture process in the fingerprint readingapparatus according to the embodiment of the present invention.

[0031]FIG. 9 is a flowchart of an image capture timing determinationprocess in the fingerprint reading apparatus according to the embodimentof the present invention.

[0032]FIG. 10 is a developed, plan view of the printing pattern of theroller in the fingerprint reading apparatus according to the embodimentof the present invention.

[0033]FIGS. 11A and 11B are illustrations of variations in the positionof a printing portion in a rotation sensing image in the fingerprintreading apparatus according to the embodiment of the present invention.

[0034]FIGS. 12A to 12D are illustrations of movement of a printingportion in a rotation sensing image in the fingerprint reading apparatusaccording to the embodiment of the present invention.

[0035]FIGS. 13A and 13B are diagrams showing another example of theprinting pattern in the fingerprint reading apparatus according to theembodiment of the present invention.

[0036]FIG. 14 is a diagram showing another example of the printingpattern in the fingerprint reading apparatus according to the embodimentof the present invention.

[0037]FIG. 15 is a flowchart illustrating a fingerprint recognitionprocess performed by the fingerprint reading apparatus according to theembodiment of the present invention.

[0038]FIGS. 16A and 16B are illustrations each showing an example of howto move a finger when a fingerprint image is captured by the fingerprintreading apparatus according to the embodiment of the present invention.

[0039]FIG. 17 is a diagram of an example of fingerprint image data readby a line sensor of the fingerprint reading apparatus according to theembodiment of the present invention.

[0040]FIG. 18 is a diagram of an arrangement of pixels (pixel values) offingerprint image data read by each image pickup element of a linesensor in the fingerprint reading apparatus according to the embodimentof the present invention.

[0041]FIG. 19 is a flowchart illustrating in detail an image datacorrecting process in the fingerprint reading apparatus according to theembodiment of the present invention.

[0042]FIG. 20 is a flowchart of a first modification to step A1 in theflowchart shown in FIG. 19.

[0043]FIG. 21 is a flowchart of a second modification to step A1 in theflowchart shown in FIG. 19.

DETAILED DESCRIPTION OF THE INVENTION

[0044]FIG. 1 is a block diagram of an electronic circuit of a mobilephone with an image reading apparatus according to an embodiment of thepresent invention.

[0045] The mobile phone has a computer which reads programs recorded ona recording medium and whose operation is controlled by the programs. Itincludes a CPU 10, a storage unit 12, a RAM 14, a communication unit 16,a display unit 18, a key unit 19 and a fingerprint reading unit 20. TheCPU 10 is connected to these units through buses. The fingerprintreading unit 20 reads an image of a fingerprint of a user's fingertip asa subject.

[0046] The CPU 10 executes programs stored in a program area of the RAM14 to fulfill various functions. It not only controls the functions ofthe mobile phone but also controls the fingerprint reading unit 20 toread image data of a fingerprint and performs various processes for theimage data.

[0047] The storage unit 12 stores programs, data and the like. When theneed arises, the programs and data are read out of the unit 12 andstored in the RAM 14.

[0048] The RAM 14 stores programs and various data items to which theCPU 10 gains access. More specifically, the RAM 14 stores programs forprocessing image data of a fingerprint read by the fingerprint readingunit 20 and performing a fingerprint recognition process as well asvarious programs for controlling the mobile phone. When the unit 20reads fingerprint image data, the RAM 14 stores the read image data.

[0049] The communication unit 16 carries out radio communication for themobile phone.

[0050] The display unit 18 displays various data items when the CPU 10performs the respective functions.

[0051] The key unit 19 is made up of a plurality of keys includingnumeric keys for inputting telephone numbers and function keys.

[0052] The fingerprint reading unit 20 is located on the front top ofthe mobile phone as shown in FIG. 2 in such a manner that it can easilyread image data of a fingerprint. The unit 20 includes a light source21, a lens optical system (CELFOC lens) 22, a line sensor(one-dimensional image pickup device) 24, an image pickup controlcircuit 26, an A/D conversion circuit 28 and a transparent, cylindricalroller 29. Part of the outer surface of the roller 29 is exposed tooutside from a slit formed in the housing of the mobile phone. A user'sfinger touches with the exposed part of the roller 29. In order to reada fingerprint of the finger, the roller 29 is rotated while the fingertouches with the outer surface of the roller 29 and moves in a givedirection (which is perpendicular to the rotation axis of the roller29). The slit has only to have such a greater width that a user canpress his or her finger on the roller 29 and rotate the roller 29. It isthus unnecessary to secure a reading area covering the entirefingerprint, and the area for the fingerprint reading unit 20 (roller29) occupied in the surface of the housing can be decreased.

[0053] In the fingerprint reading unit 20, the light source 21 radiateslight. The light is reflected by the finger touching with (contacting)the roller 29 and then condensed on the line sensor 24 through theCELFOC lens 22. The line sensor 24 photoelectrically converts thecondensed light into a signal under the control of the image pickupcontrol circuit 26. The A/D conversion circuit 28 converts the signalinto image data indicating a fingerprint. The line sensor 24 reads imagedata periodically, e.g., 20000 times per second. The image data isbuffered in the RAM 14. The CPU 10 extracts a printing pattern 30printed on the roller 29 from the images read by the line sensor 24 as arotation sensing image. Based on variations in the rotation sensingimage, the CPU 10 captures image data suitable for forming a fingerprintimage from the periodically read image data and then records it in theRAM 14.

[0054]FIG. 3 is a schematic sectional view of a mechanical section ofthe fingerprint reading unit 20.

[0055] The housing of the mobile phone has a slit through which part ofthe outer surface of the roller 29 is exposed. The slit is formed alongthe rotation axis of the roller 29. The roller 29 is made of transparentmaterials that transmit light, such as acryl and glass. The roller 29can rotate with its outer surface partly exposed to outside from theslit. The roller 29 has a hollow containing an image pickup functionalunit that is made up of the light source 21, lens optical system (CELFOClens) 22, and line sensor 24. This unit does not interlock with therotation of the roller 29.

[0056] The CELFOC lens 22 forms on the line sensor 24 an image of aportion of the roller 29 with which a finger touches. The presentembodiment is not limited to the CELFOC lens. It can be applied to animage-forming optical system including a rod lens group.

[0057] The line sensor 24 is a CCD line sensor, a CMOS line sensor orthe like. The line sensor 24 includes a plurality of image pickupelements that are arranged in parallel to the rotation axis of theroller 29. The line sensor 24 has an image pickup range corresponding tothe length of the roller 29. It can read an image including the printingpattern 30 that is printed on the roller 29 at one end (or near oneend). Since the hollow of the roller 29 includes the image pickupfunctional unit, the area for the fingerprint reading unit 20 occupiedin the housing can be decreased, as can be the volume of the unit 20therein.

[0058]FIG. 4 is a perspective view showing an outward appearance of theroller 29.

[0059] The printing pattern 30 is formed on the entire circumference ofthe roller 29. The printing pattern 30 has a plurality of segments thatare sloped with respect to the rotation axis of the roller 29 at thesame angle. The segments have the same length. If the roller 29 isdeveloped into a plane, the segments are arranged at regular intervalsand in parallel to each other. Adjacent segments of the pattern 30overlap each other in the reading direction (see FIG. 10). Assume thatthe printing pattern 30 is obtained by forming black segments on a whitebackground. In a correcting data generation process that will bedescribed later, the white background is used as the reference of whiteimages, whereas the segments are used as the reference of black images.

[0060] The light source 21 may include an LED, a fluorescent tube, ahalogen lamp and the like.

[0061]FIG. 5 shows a correspondence between the printing pattern 30printed on the roller 29 and image pickup elements of the line sensor24. The line sensor 24 has an image pickup range corresponding to thewhole length of the roller 29. The number (N) of image pickup elementscorresponding to the printing pattern 30 has been known and they areused for various processes described later.

[0062] Here are specific numeric values. When the diameter of the roller29 is 7 mm and the resolution of the line sensor 24 is 600 dpi, theprinting pattern 30 is formed at a segment angle of 45° and a segmentinterval of 40 dots (the length of forty image pickup elements of theline sensor 24, i.e., about 1.7 mm) and with a segment width of 2 dots(about 0.08 mm) and a diagonal line (about 2.5 mm in length) of twosegments each having a length of 60 dots.

[0063] A fingerprint image reading operation performed by thefingerprint reading unit 20 will be described with reference to FIGS. 6to 9.

[0064] The fingerprint reading unit 20 reads image data continuouslyfrom the line sensor 24 20000 times per second irrespective of therotation of the roller 29 or the angle of the rotation. The image dataincludes the printing pattern 30 printed on the roller 29 as well as afingerprint image (see FIG. 12).

[0065] The CPU 10 detects an amount of rotation of the roller 29 basedon the amount of shift of black pixels (segments) included in the imagedata read by the line sensor 24, and stores image data necessary forforming a fingerprint image as fingerprint image data.

[0066]FIG. 6 is a main flowchart of the entire process performed eachtime the line sensor 24 captures image data.

[0067] When image data of one line is read out of the line sensor 24,the CPU 10 generates correcting data (step A1). The correcting data is apixel value representing black and white pixels based on which arotation sensing image is recognized. The CPU 10 extracts a rotationsensing image using the correcting data (step A2). The CPU 10 determinesimage capture timing based on the amount of shift of the rotationsensing image (step A3) and then determines whether the image data isnecessary for forming a fingerprint image (step A4). If now is the imagecapture timing, the image data of one line is captured as an image forforming a fingerprint (step A5).

[0068] There now follows a detailed description of each of the steps.

[0069]FIG. 7 is a flowchart showing a correcting data generation processin detail.

[0070] Of the pixel values of pixels that each image pickup element hasread so far, the maximum value is recorded as a white reference valueand the minimum value is recorded as a black reference value. Thesevalues are recorded as correcting data. As the roller 29 rotates, theimage pickup element captures data (maximum and minimum values) that areconsidered to be white and black reference values.

[0071] Assume here that the number of image pickup elements used forreading a rotation sensing image is N, the maximum and minimum values ofpixels that each image pickup element has read so far are{max_(i)|0≦i≦n−1}, {min_(i)|0≦i≦N−1}, respectively, and data that eachimage pickup element newly reads is {data_(i)|0≦i≦N−1}.

[0072] When the CPU 10 reads pixels (pixel values) from the printingpattern 30 read by the line sensor, it initializes i to 0 (i=0) (stepB1). The CPU 10 compares the maximum pixel value max_(i) that the i-thimage pickup element has read so far and the currently-noted pixel valuedata_(i) (step B2). If max_(i)<data_(i), the pixel value data_(i) isupdated to a new maximum value max_(i) (step B3).

[0073] The CPU 10 also compares the minimum pixel value min_(i) that thei-th image pickup element has read so far and the currently-noted pixelvalue data_(i) (step B4). If min_(i)>data_(i), the currently-noted pixelvalue data_(i) is updated to a new minimum value min_(i) (step B5).

[0074] The CPU 10 updates i to i+1 (i=i+1) (step B6). If i<N and allimage pickup elements have not yet completed their operation for theprinting pattern 30 (step B8), the CPU confirms whether the maximum andminimum values max_(i) and min_(i) should be updated by the pixel valuedata_(i) read by the next i-th image pickup element in the same manneras described above.

[0075] The above process is carried out for the pixel values read by theimage pickup elements 0 to N−1.

[0076]FIG. 8 is a flowchart showing a rotation sensing image extractionprocess in detail.

[0077] In this process, the CPU 10 determines whether pixel data (pixelvalue) read by each image pickup element is black or white. Thisdetermination can be achieved on the basis of a threshold value that isobtained from the maximum and minimum values of the pixel data that theimage pickup element has read so far. The image determined as black isprocessed as a rotation sensing image.

[0078] In this embodiment, the CPU 10 determines whether each pixel isblack or white by considering an average of the maximum and minimumvalues to be a threshold value.

[0079] First, the CPU 10 initializes i to zero (i=0) (step C1). Itcalculates the sum of the maximum and minimum values max_(i) and min_(i)that the i-th image pickup element has so far read and considers halfthe sum to be a threshold value th (step C2). It compares the thresholdvalue with the i-th pixel value data_(i) (step C3). If th<data_(i) (YESin step C3), the CPU 10 determines the pixel as white (step C4). Ifth≧data_(i) (NO in step C3), the CPU 10 determines the pixel as black(step C5).

[0080] Then, the CPU 10 updates i to i+1 (step C6). If i<N and eachimage pickup element has not yet completed its operation for therotation sensing image (step C7), the CPU 10 determines whether thepixel value data_(i)read by the next i-th image pickup element is blackor white in the same manner as described above.

[0081] The above process is carried out for the pixel values read by theimage pickup elements 0 to N−1. The CPU 10 can thus determine whethereach pixel is black or white based on the average of the maximum andminimum values that have been so far read by each image pickup element.The black pixel can be extracted as a rotation sensing image.

[0082]FIG. 9 is a flowchart showing a timing determination process indetail.

[0083] First, the CPU 10 senses one or two printing portions (blackpixels) corresponding to segments of printing patterns 30 from the imagedetermined as black by the rotation sensing image extraction process(step D1).

[0084] The CPU 10 compares a location of the sensed printing portionwith that of the printing portion sensed last time with image capturetiming (step D2). If the printing portions are separated at more than agiven distance, the CPU 10 determines that now is image capture timing.

[0085] Since the printed segments (black portions) have a width, theline sensor 24 senses black portions of a plurality of pixels. In thiscase, the CPU 10 compares the middle of a row of black pixels or one endof the row as a printing portion. If there are a plurality of printingportions sensed last time with the image capture timing and those sensedthis time, the printing portions are compared for all combinations ofthe printing portions.

[0086]FIG. 10 is a developed, plane view of the printing pattern 30 ofthe roller 29. The line sensor 24 reads an image in parallel to therotation axis of the roller 29, or in the horizontal direction in FIG.10. The segments (oblique lines) for forming the printing pattern 30 arearranged such that their adjacent segments overlap in a range defined bybroken lines a and b in FIG. 10. The line sensor 24 can sense one or twoportions of the printing pattern 30 by a single operation.

[0087]FIG. 11A shows a rotation sensing image that is read with the lastimage capture timing. Two printing portions a1 and a2 are sensed fromthe image. If another two printing portions b1 and b2 are sensed at thistime as shown in FIG. 11B, the CPU 10 compares printing portions foreach of four combinations a1 and b1, a1 and b2, a2 and b1, and a2 andb2.

[0088] Even though the line sensor 24 reads one end of a segment, it isto read an adjacent segment at once. Though the reading accuracy of theone end of the segment is low, the distance of printing portions isdetermined by all combinations of printing portions and thus therotation of the roller 29 can stably be sensed on the basis of printingportions corresponding to the adjacent segment that is read with highprecision.

[0089] If the minimum distance of all the combinations of printingportions is a given value or larger and the shift of the printingportions is confirmed (YES in step D3), the CPU 10 determines that nowis image capture timing (step D4). In other words when the roller 29 isrotated with a subject pressed thereon and a new portion of the subjectshifts to a reading position of the line sensor 24, the image read bythe line sensor this time is captured as a fingerprint image. If theminimum distance is not larger than a given value (NO in step D3), theCPU 10 determines that now is not image capture timing (step D5).

[0090] In the above configuration, the diameter of the roller 29 is 7mm, the resolution of the line sensor 24 is 600 dpi, the angle of eachsegment is 45°, the interval of segments is 40 dots, the width of eachsegment is 2 dots, and the length of each diagonal line is 60 dots. Theamount of shift of a printing portion, which corresponds to imagecapture timing, can be set at one dot (0.04 mm).

[0091] If the CPU 10 determines the image capture timing through theimage capture timing determination process (step E6), it records theimage read by the line sensor 24 as a fingerprint image.

[0092] If the CPU 10 determines that now is not image capture timing, itabandons the image read by the line sensor 24.

[0093] The CPU 10 can determine image capture timing based on the shiftof the locations of printing portions corresponding to the segments ofthe printing pattern 30 that is a rotation sensing image and generate afingerprint image. Since the rotation sensing image is corrected inresponse to variations in outside light and light source, image capturetiming can be determined with stability.

[0094] The segments of the printing pattern 30 are printed in parallelwith each other at a given angle with respect to the rotation axis ofthe roller 29. The read portions of the printed segments shift right orleft according to the direction in which the roller 29 rotates. In otherwords, the rotating direction of the roller 29 can be sensed by thedirection in which the portions of the segments shift.

[0095] In the printing pattern 30 shown in FIG. 10, a user moves his orher finger in the direction A (toward the user) as shown in FIG. 3 andthe roller 29 rotates in the direction B accordingly. The printingportion shifts in the left direction from that of the previously readrotation sensing image. If the user moves his or her finger away fromhim or her and the roller 29 rotates, the printing portion shifts in theright direction from that of the previously read rotation sensing imageas shown in FIGS. 12A to 12D.

[0096] The CPU 10 can detect a direction in which the roller 19 rotatesand determine whether an image is read from the lower part of afingerprint or the upper part thereof (fingertip). The CPU 10 can recordan image read with read timing in sequence from the upper or lower partof the fingerprint and generate a fingerprint image in a given direction(e.g., the fingertip is set upward). If the CPU 10 determines that animage is read from the lower part of the fingerprint, it can reverse thefingerprint 180 degrees after it reads all the images recorded insequence.

[0097] As described above, the maximum and minimum values of the pixelvalues that have so far read are recorded for each of the image pickupelements of the line sensor 24, and the average of these values isconsidered to be a reference value. Since the CPU 10 recognizes arotation sensing image based on the reference value, it can determineimage capture timing with stability even though the image pickupelements vary in precision and the outside light and light sourcechange. The segments of the printing pattern 30 printed on the roller 29are arranged to overlap each other when the line sensor 24 captures animage. Even though one end of a single segment is a reading position ofthe line sensor 24, the line sensor 24 reads an image halfway in anothersegment; therefore, image capture timing can stably be determinedwithout decreasing in precision.

[0098] The above printing pattern 30 is obtained by printing blacksegments on a white background. However, a portion other than the blacksegments can be printed. White segments can be printed on a blackbackground and, in this case, they have only to be processed in the samemanner as done for the black segments.

[0099] The segments of the printing pattern 30 are thickened to read arotation sensing image as shown in FIG. 13A and sense edge portionswhere white changes to black or black changes to white as shown in FIG.13B. Image capture timing can thus be determined according to variationsin locations of the edge portions.

[0100] A saw-tooth pattern (isosceles triangles of the same shape arearranged adjacent to each other in the same direction) can be printed oneither end of the roller 29 as shown in FIG. 14. The pattern is printedsuch that when the side of a triangle (which is parallel to the readingdirection) at one end of the roller 29 is a reading position, the slantside of a triangle at the other end thereof becomes a reading position.In reading position a in FIG. 14, for example, the side of a triangle atthe right end of the roller 29 becomes a reading position, which willprobably decrease in precision. The slant side of a triangle at the leftend of the roller 29 and thus the rotation of the roller 29 can besensed with precision in accordance with variations in the position ofthe slant side.

[0101] The printing pattern 30 is printed on the roller 29. However, thesurface of the roller 29 can be scratched and deformed to form apattern, some materials can be adhered to the surface, or the surfacecan be melted. Any pattern will be formed on the surface of the roller29 if the position of the image read by the image sensor 24 can bedetected.

[0102] The image reading timing determination process using a printingpattern has been described so far. There now follows descriptions of aprocess of adjusting a pixel value when a fingerprint image is captured.

[0103]FIG. 15 is a flowchart illustrating a fingerprint recognitionprocess of a mobile phone.

[0104] When the CPU 10 receives an instruction to perform a fingerprintrecognition process from, e.g., the key unit 19, it executes afingerprint recognition program to start the process. The CPU 10 thenperforms a fingerprint image reading process to read a fingerprint imageto be recognized through the fingerprint reading unit 20 (step S1).

[0105]FIGS. 16A and 16B illustrate an example of how to move a fingerwhen the fingerprint reading unit 20 reads a fingerprint image.

[0106] In FIG. 16A, a user's fingertip touches with a reading portion onthe roller 29 and in this condition the roller 29 rotates and moves in agiven direction (which is perpendicular to the rotation axis of theroller 29). In FIG. 16B, the fingertip moves toward the user. The userrotates the roller 29 while moving his or her fingertip. The line sensor24, which is arranged in the roller 29 in parallel to the rotation axisof the roller 29, can scan a fingerprint portion of the fingertip.

[0107] The A/D conversion circuit 28 converts an image signal outputfrom the line sensor 24 into digital data (fingerprint image data). TheRAM 41 stores the fingerprint image data through the fingerprint readingunit 20. The fingerprint image data has a multilevel pixel value (e.g.,0 to 255). The fingerprint image data read by the unit 20 by opticalscanning may include bias and variations in the pixel values under theinfluence of variations in the image pickup elements of the line sensor24 and variations in the CELFOC lens (lens optical system) 22.

[0108] In order to correct the above bias and variations, the CPU 10corrects the fingerprint image data read by the fingerprint reading unit20 (step S2). The CPU 10 corrects the respective pixel values of thefingerprint image data.

[0109]FIG. 19 is a flowchart showing an image data correcting process indetail.

[0110] The fingerprint image data read by the line sensor 24 includes aplurality of rows of data items, corresponding to each of the imagepickup elements of the line sensor 24, that are arranged in the y-axisdirection as shown in FIG. 17. The rows of data items are similar to oneanother in property. A fingerprint is made up of ridges and valleys andhas nearly a uniform pattern. The data items of the respective rows donot widely differ in average or property (e.g., ratio of the number ofpixels having each pixel value to the total number of pixels). The CPU10 thus corrects the pixel values of the fingerprint image data usingits property as will be described below.

[0111] First, the CPU 10 detects the maximum pixel value max_(i) (firstpixel value) and minimum pixel value min_(i) (second pixel value) fromimage data read by each of the image pickup elements (step T1). FIG. 18shows the pixels (pixel values) of image data D read by each of theimage pickup elements. D={d_(ij)|i=1 . . . n, j=1 . . . m}. The maximumpixel value max_(i) and minimum pixel value min_(i) are given by thefollowing equations (1) and (2):

max_(i)=MAX(d_(ij) |j=1 . . . m), 1≦i≦n  (1)

min_(i)=MIN(d_(ij) |j=1 . . . m), 1≦i≦n  (2)

[0112] where MAX( ) and MIN( ) are functions for obtaining the maximumand minimum pixel values.

[0113] Then, the CPU 10 calculates the width of data for each imagepickup element, or the range between the maximum and minimum pixelvalues obtained in step T1, by the following equation (3) (step T2):

range_(i)=max_(i)−min_(i), 1≦i≦n  (3).

[0114] The CPU 10 calculates normalized data d′_(ij) by the followingequation (4) (step T3). The normalized data d′_(ij) represents the ratioof pixel value d_(ij) to pixel value range range_(i) for the pixelcaptured by each of the image pickup elements of the line sensor 24.

d′ _(ij)=(d _(ij)−min_(i))/range_(i);

1≦i≦n, 1≦j≦m  (4)

[0115] The CPU 10 calculates an average of normalized data d′_(ij) bythe following equation (5) (step T4): $\begin{matrix}{{{ave}_{i} = {\frac{1}{m}{\sum\limits_{j = 1}^{m}d_{i\quad j}^{\prime}}}};{1 \leqq i \leqq n}} & (5)\end{matrix}$

[0116] The CPU 10 obtains pixel data using the average of the normalizeddata d′_(ij) (step T5). More specifically, the CPU 10 corrects a pixelvalue of each pixel of image data read by each of the image pickupelement based on the average (ave_(i)) of normalized data d′_(ij)calculated in step T4 and the pixel value (depth) of the pixel, by thefollowing equation (6): $\begin{matrix}{{{d_{i\quad j}^{''} = {d_{i\quad j}^{\prime} \times \frac{depth}{2 \times {ave}_{i}}}};{1 \leqq i \leqq n}},{1 \leqq j \leqq m}} & (6)\end{matrix}$

[0117] where depth is the maximum pixel value of one pixel (e.g., 255 ifone pixel is 8 bits).

[0118] When a pixel value is obtained by the proportional calculation ofthe above equation (6), it may exceed the maximum or minimum pixel valueof one pixel. In this case, a clipping operation is performed to adjustthe pixel value so as to fall within a range of the pixel value of onepixel.

[0119] The CPU 10 normalizes the fingerprint image data corrected by theimage data correcting process to align its slope and size with those ofthe registered fingerprint image data of the original data (step S3).

[0120] The CPU 10 compares the normalized fingerprint image data withthe registered fingerprint image data by the fingerprint comparingprocess (step S4). Since a fingerprint image to be compared in the stageprecedent to the fingerprint comparing process is corrected,satisfactory comparing results can be expected.

[0121] The CPU 10 displays the results (OK, NG) of the fingerprintcomparing process on, e.g., the display unit 18 (step S5).

[0122] Performing the above image data correcting process, the CPU 10can correct the fingerprint image data read by the line sensor 24 toeliminate an influence based on variations in the image pickup elementsand the CELFOC lens (lens optical system) 22. Since no data need to beprepared for adjustment in advance, no pre-scanning is required forimage data correction. Since an image is corrected in response tovariations in environment such as outside light when the image is read,fingerprint comparing results can always be obtained with stabilityregardless of the environment.

[0123] There now follows an explanation as to modifications (1) and (2)to the above embodiment.

[0124] Modification (1)

[0125] If, for example, dust is attached to the surface of the roller 29when a fingerprint image is read, the dust will have a pixel valuedifferent from the normal pixel value of a fingerprint and thus thepixel value may correspond to the maximum or minimum pixel value. Nofingerprint image can be accurately corrected due to the influence ofthe pixel value of the dust. In the corrected fingerprint image, the rowof data items corresponding to the dust differs from the other rows inpixel value, with the result that a pattern such as a stripe (or a line)will be formed.

[0126] In the foregoing embodiment, the maximum and minimum pixel valuesare detected from the pixels captured by the image pickup elements ofthe line sensor 24. In the modification (1), however, the pixel valuesof pixels arranged in designated order are detected from the maximum andminimum pixel values. Image data can thus be corrected without anyinfluence of pixels having incorrect pixel values due to dust or thelike.

[0127]FIG. 20 is a flowchart showing in detail a step of themodification (1) which corresponds to the step T1 of the flowchart shownin FIG. 19. Since the other steps are the same as those of the imagedata correcting process shown in FIG. 19, their descriptions areomitted.

[0128] Assume in the modification (1) that the storage unit 12 stores adesignated value indicating the order of pixels of image data capturedby each image pickup element. The designated value can be recorded inthe storage unit 12 in advance or through the key unit 19 by a user. Forexample, the storage unit 12 stores the how-manieth pixel (e.g., x) froma pixel with the maximum value in the pixels read by each image pickupelement and the how-manieth pixel (e.g., y) from a pixel with theminimum value therein.

[0129] A group of pixels read by a first image pickup element of theline sensor 24 is designated (step U1). The pixels are sorted (inascending or descending numeric order) based on their pixel values (stepU2).

[0130] The x-th largest pixel value is detected as the maximum pixelvalue for generating normalized data, while the y-th smallest pixelvalue is detected as the minimum pixel value for generating normalizeddata (step U3).

[0131] The next image pickup element for detecting the minimum andmaximum values is designated (step U4). Similarly, the x-th largestpixel value and the y-th smallest pixel value are detected from thegroup of pixels read by the image pickup element (steps U5, U2 and U3).

[0132] Image data items read by each of the image pickup elements aresorted when their pixels have a pixel value {d_(ij)|i=1 . . . n, j=1 . .. m}. Of the data items {d_(ij)|j=1 . . . m} captured by the i-th imagepickup element, the image data item with the maximum pixel value issd_(i1) and the image data item with the second maximum pixel value issd_(i2). If these image data items sorted in descending order arerepresented as {sd_(ik)|k=1 . . . m}, the maximum and minimum valuesmax_(i) and min_(i) for normalizing data for each image pickup elementare calculated as follows.

max_(i) =sd _(ix)  (7)

min_(i) =sd _(i(m-y+1))  (8)

[0133] According to the modification (1), even though there are pixelswith a pixel value that is not normally obtained from a fingerprintimage due to, for example, dust attached to the roller 29, they can beeliminated by obtaining the maximum and minimum values max_(i) andmin_(i) of the range of the pixel value to normalize data. The influenceof the dust can thus be eliminated to correct image data.

[0134] Modification (2)

[0135] In the above modification (1), the x-th and y-th pixel value fromthe maximum and minimum values are detected from the pixels read by eachimage pickup element. However, the maximum and minimum pixel values forgenerating normalized data can be obtained from a group of pixels withthe maximum value to the designated-manieth pixel and a group of pixelswith the minimum value to the designated-manieth pixel. Assume here thatthe average of the former group of pixels is the maximum value and thatof the latter group of pixels is the minimum value.

[0136]FIG. 21 is a flowchart showing in detail a step of themodification (2) which corresponds to the step T1 of the flowchart shownin FIG. 19. Since the other steps are the same as those of the imagedata correcting process shown in FIG. 19, their descriptions areomitted. Assume in the modification (2) that the storage unit 12 storesa designated value indicating the order of pixels of image data capturedby each image pickup element, as in the modification (2).

[0137] A group of pixels read by a first image pickup element of theline sensor 24 is designated (step V1). The pixels are sorted (inascending or descending numeric order) based on their pixel values (stepV2).

[0138] The average of pixel values of a pixel with the maximum value tothe x-th pixel is detected as the maximum pixel value for generatingnormalized data, while the average of pixel values of a pixel with theminimum value to the y-th pixel is detected as the minimum pixel valuefor generating normalized data (step V3).

[0139] The next image pickup element for detecting the maximum andminimum values is designated (step V4). Similarly, the average of pixelvalues of the pixel with the maximum value to the x-th pixel and that ofpixel values of the pixel with the minimum value to the y-th pixel aredetected from the group of pixels read by the image pickup element(steps U5, U2 and U3). These averages are considered to be the maximumand minimum values (steps V5, V2 and V3).

[0140] For example, the average of pixel values of the pixel with themaximum value to the x-th pixel and that of pixel values of the pixelwith the minimum value to the y-th pixel are given by the followingequations (9) and (10), respectively. $\begin{matrix}{{\max_{i}{= {\frac{1}{x}{\sum\limits_{j = 1}^{x}{s\quad d_{i\quad j}}}}}};{1 \leqq i \leqq n}} & (9) \\{{{mix}_{i} = {\frac{1}{y}{\sum\limits_{j = {m - y + 1}}^{m}{s\quad d_{i\quad j}}}}};{1 \leqq i \leqq n}} & (10)\end{matrix}$

[0141] According to the modification (2), good results can stably beobtained by calculating the maximum and minimum pixel values.Fingerprint image data captured by each image pickup element includes animage portion corresponding to a number of ridges and valleys of afingerprint. Of a set (group) of pixels having a pixel with the maximumor minimum pixel value through a designated-manieth pixel, there are anumber of pixels whose pixel values are close to one another. Eventhough dust is attached to the roller 29, the influence of pixelscorresponding to the dust can be lessened by obtaining the average ofthe group of pixels.

[0142] In the foregoing descriptions, the fingerprint image data readingapparatus is applied to mobile phones. It can be applied to otherinformation devices or used alone.

[0143] In the foregoing descriptions, the fingerprint image data readingapparatus is used to read a fingerprint image. It can read image datasuch as a palm print pattern. The fingerprint reading unit 20 (linesensor 24) can read a palm print image if the pixel values of pixels ofthe palm print image, which are arranged in the y-axis direction, aresimilar to each other in average and maximum and minimum pixel valuesthereof between the image pickup elements of the unit 20 as in the caseof the above fingerprint image.

[0144] Additional advantages and modifications will readily occur tothose skilled in the art. Therefore, the invention in its broaderaspects is not limited to the specific details and representativeembodiments shown and described herein. Accordingly, variousmodifications may be made without departing from the spirit or scope ofthe general inventive concept as defined by the appended claims andtheir equivalents.

What is claimed is:
 1. An image reading apparatus which reads afingerprint image, comprising: a roller rotatably mounted to the imagereading apparatus, a given pattern being printed on part of an outersurface of the roller in a circumferential direction; a line sensorincluding a plurality of image pickup elements which read a fingerprintimage from a finger that touches with the roller as well as the patternprinted on the roller; a generating unit which generates a referencevalue to each of the image pickup elements corresponding to part of theline sensor, which reads the pattern, based on image data of the patternread by the line sensor; a rotation sensing image extracting unit whichextracts a rotation sensing image based on the reference value generatedby the generating unit; and a determination unit which determinescapture timing of the fingerprint image read by the line sensor based onvariations in the rotation sensing image extracted by the rotationsensing image extracting unit.
 2. The image reading apparatus accordingto claim 1, wherein the pattern has a plurality of segments formed at asame angle with respect to a rotation axis of the roller, and thesegments are arranged such that an image of one end of a segment and animage of another segment are simultaneously read by the line sensor. 3.The image reading apparatus according to claim 1, wherein the generatingunit generates maximum and minimum pixel values of pixels read by theline sensor so far in accordance with rotation of the roller as areference value for each of the image pickup elements of the linesensor.
 4. The image reading apparatus according to claim 1, furthercomprising a rotation direction determining unit which determines arotation direction of the roller based on the variations in the rotationsensing image; and an image recording unit which records an image withthe capture timing determined by the determination unit in accordancewith the rotation direction determined by the rotation directiondetermining unit.
 5. An image processing apparatus comprising: a linesensor including a plurality of image pickup elements; a pixel valuedetecting unit which detects a first pixel value and a second pixelvalue from each image data including a plurality of pixels output fromeach of the image pickup elements; a pixel value range detecting unitwhich detects a pixel value range between the first pixel value and thesecond pixel value detected by the pixel value detecting unit; anormalized data generating unit which generates normalized data thatindicates a ratio of a pixel value of each of the pixels of the imagedata to the pixel value range; a normalized data average calculatingunit which calculates an average of the normalized data generated by thenormalized data generating unit; and a pixel value correcting unit whichcorrects a pixel value of each of the pixels of the image data based onthe average calculated by the normalized data average calculating unitand the pixel value of each of the pixels.
 6. The image processingapparatus according to claim 5, wherein the pixel value detecting unitdetects a maximum pixel value and a minimum pixel value from the imagedata output from each of the image pickup elements as the first pixelvalue and the second pixel value, respectively.
 7. The image processingapparatus according to claim 5, wherein the pixel value detecting unitincludes: a designated value recording unit which records a designatedvalue that indicates an order of the pixels of the image data outputfrom the image pickup elements; a first detection unit which detects apixel value of a pixel of the image data output from the image pickupelements as the first pixel value, the pixel value being adesignated-manieth largest one recorded by the designated valuerecording unit; and a second detection unit which detects a pixel valueof a pixel of the image data output from the image pickup elements asthe second pixel value, the pixel value being a designated-maniethsmallest one recorded by the designated value recording unit.
 8. Theimage processing apparatus according to claim 5, wherein the pixel valuedetecting unit includes: a designated value recording unit which recordsa designated value that indicates an order of the pixels of the imagedata output from the image pickup elements; a first setting unit whichsets an average of pixel values from a maximum pixel value to adesignated-manieth pixel value recorded by the designated valuerecording unit in the image data output from the image pickup elementsas the first pixel value; and a second setting unit which sets anaverage of pixel values from a minimum pixel value to adesignated-manieth pixel value recorded by the designated valuerecording unit in the image data output from the image pickup elementsas the second pixel value.
 9. The image processing apparatus accordingto claim 5, wherein the line sensor is fixed in a transparent, hollowroller that is rotatably mounted to the image processing apparatus. 10.The image processing apparatus according to claim 9, wherein the linesensor reads a fingerprint image of a finger that is in contact with theroller.
 11. A fingerprint recognition apparatus comprising: a linesensor including a plurality of image pickup elements which capturefingerprint image data for each of a plurality of pixels; a pixel valuedetecting unit which detects a first pixel value and a second pixelvalue from the fingerprint image data captured by each of the imagepickup elements; a pixel value range detecting unit which detects apixel value range between the first pixel value and the second pixelvalue; a normalized data generating unit which generates normalized datathat indicates a ratio of a pixel value of each of the pixels of thefingerprint image data to the pixel value range; a normalized dataaverage calculating unit which calculates an average of the normalizeddata generated by the normalized data generating unit; a pixel valuecorrecting unit which corrects a pixel value of each of the pixels ofthe fingerprint image data based on the average calculated by thenormalized data average calculating unit and the pixel value of each ofthe pixels; and a fingerprint recognizing unit which recognizes thefingerprint image data whose pixel value is corrected by the pixel valuecorrecting unit.
 12. The fingerprint recognition apparatus according toclaim 11, wherein the pixel value detecting unit detects a maximum pixelvalue and a minimum pixel value from the fingerprint image data capturedby each of the image pickup elements as the first pixel value and thesecond pixel value, respectively.
 13. The fingerprint recognitionapparatus according to claim 11, wherein the pixel value detecting unitincludes: a designated value recording unit which records a designatedvalue that indicates an order of the pixels of the fingerprint imagedata output from the image pickup elements; a first detection unit whichdetects a pixel value of a pixel of the fingerprint image data outputfrom the image pickup elements as the first pixel value, the pixel valuebeing a designated-manieth largest one recorded by the designated valuerecording unit; and a second detection unit which detects a pixel valueof a pixel of the fingerprint image data output from the image pickupelements as the second pixel value, the pixel value being adesignated-manieth smallest one recorded by the designated valuerecording unit.
 14. The fingerprint recognition apparatus according toclaim 11, wherein the pixel value detecting unit includes: a designatedvalue recording unit which records a designated value that indicates anorder of the pixels of the fingerprint image data output from the imagepickup elements; a first setting unit which sets an average of pixelvalues from a maximum pixel value to a designated-manieth pixel valuerecorded by the designated value recording unit in the fingerprint imagedata output from the image pickup elements as the first pixel value; anda second setting unit which sets an average of pixel values from aminimum pixel value to a designated-manieth pixel value recorded by thedesignated value recording unit in the fingerprint image data outputfrom the image pickup elements as the second pixel value.
 15. Thefingerprint recognition apparatus according to claim 11, wherein theline sensor is fixed in a transparent, hollow roller that is rotatablymounted to the fingerprint recognition apparatus.
 16. A method ofprocessing image data which is captured by a line sensor including aplurality of image pickup elements and whose pixels each have amultilevel pixel value, the method comprising: detecting a first pixelvalue and a second pixel value from the image data captured by each ofthe image pickup elements of the line sensor; detecting a pixel valuerange between the first pixel value and the second pixel value;generating normalized data that indicates a ratio of a pixel value ofeach of the pixels of the image data to the pixel value range;calculating an average of the normalized data; and correcting a pixelvalue of each of the pixels of the image data based on the average andthe pixel value of each of the pixels.
 17. The method according to claim16, wherein the pixel value detecting detects a maximum value and aminimum value from the image data output from each of the image pickupelements as the first pixel value and the second pixel value,respectively.
 18. The method according to claim 16, wherein the pixelvalue detecting includes: recording a designated value that indicates anorder of the pixels of the image data output from the image pickupelements; detecting a pixel value of a pixel of the image data outputfrom the image pickup elements as the first pixel value, the pixel valuebeing a designated-manieth largest one recorded by the designated valuerecording unit; and detecting a pixel value of a pixel of the image dataoutput from the image pickup elements as the second pixel value, thepixel value being a designated-manieth smallest one recorded by thedesignated value recording unit.
 19. The method according to claim 16,wherein the pixel value detecting includes: recording a designated valuethat indicates an order of the pixels of the image data output from theimage pickup elements; setting an average of pixel values from a maximumpixel value to a designated-manieth pixel value recorded by thedesignated value recording unit in the image data output from the imagepickup elements as the first pixel value; and setting an average ofpixel values from a minimum pixel value to a designated-manieth pixelvalue recorded by the designated value recording unit in the image dataoutput from the image pickup elements as the second pixel value.
 20. Themethod according to claim 16, further comprising capturing a fingerprintimage by the line sensor.